Arithmetic Coding with Folds and Unfolds

Arithmetic coding is a method for data compression. Although the idea was developed in the 1970’s, it wasn’t until the publication of an “accessible implementation” [14] that it achieved the popularity it has today. Over the past ten years arithmetic coding has been refined and its advantages and disadvantages over rival compression schemes, particularly Huffman [9] and Shannon-Fano [5] coding, have been elucidated. Arithmetic coding produces a theoretically optimal compression under much weaker assumptions than Huffman and Shannon-Fano, and can compress within one bit of the limit imposed by Shannon’s Noiseless Coding Theorem [13]. Additionally, arithmetic coding is well suited to adaptive coding schemes, both character and word based. For recent perspectives on the subject, see [10, 12]. The “accessible implementation” of [14] consisted of a 300 line C program, and much of the paper was a blow-by-blow description of the workings of the code. There was little in the way of proof of why the various steps in the process were correct, particularly when it came to the specification of precisely what problem the implementation solved, and the details of why the inverse operation of decoding was correct. This reluctance to commit to specifications and correctness proofs seems to be a common feature of most papers devoted to the topic. Perhaps this is not surprising, because the plain fact is that arithmetic coding is tricky. Nevertheless, our aim in these lectures is to provide a formal derivation of basic algorithms for coding and decoding. Our development of arithmetic coding makes heavy use of the algebraic laws of folds and unfolds. Although much of the general theory of folds and unfolds is well-known, see [3, 6], we will need one or two novel results. One concerns a new pattern of computation, which we call streaming. In streaming, elements of an output list are produced as soon as they are determined. This may sound like lazy evaluation but it is actually quite different.